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ABSTRACT 



This thesis proposes a design for a DSS that will be used 
by the designers of instruction manuals for enlisted service 
members in the grades of E-l - E-5 in the Department of 
Defense (DoD) . The purpose of this proposed DSS is to help 
authors create manuals that will be easily comprehended by 
service members so they can quickly and effortlessly 
accomplish a task. Current research from the document design 
field and Plain English movement are reviewed to determine the 
best way to structure a written document whose sole purpose is 
adult instruction. The rules for creating the DSS are 
developed from this literature review. 
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I . INTRODUCTION 



A. BACKGROUND 

The two basic tenets of military leadership are 
accomplishing the mission and looking out for the welfare of 
the troops. The mission cannot be accomplished if the troops 
do not understand how to complete the tasks required to reach 
the objective. Since personalized individual training cannot 
be provided to each Airmen, Sailor, Soldier, and Marine, 
training manuals have been developed for completing certain 
tasks. However, these manuals are of no value if service 
members cannot access the information they need quickly and 
easily . 

This thesis examines current material from the document 
design field and Plain English movement to determine the best 
way to structure an instruction manual, so it will be useable 
and understandable. A proposed DSS to help designers reach 
this goal, will be presented. 

B. OBJECTIVES 

The purpose of this thesis is to purpose a DSS that will 
be used by the designers of instructional manuals for enlisted 
service members in the grades of E-l to E-5 in The Department 
of Defense (DoD) . The aim of this DSS is to help authors 
create manuals that will be easily comprehended by service 
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members so they can quickly and effortlessly accomplish a 
task . 

C. THE RESEARCH QUESTION 

The primary research question for this thesis is to 
determine what factors increase the readability and 
comprehensibility of instruction manuals for adults. The 
factors to be studied are organization, style, visual 
structure of the page, and information access. The secondary 
research question is to determine the best way to structure 
the DSS. 

D. SCOPE, LIMITATIONS AND ASSUMPTIONS 

Scope: This thesis focuses on established literature by 
researchers in the document design and Plain English 
movements. It does not examine experimental theories or 
methodologies in either field. 

Limitations: Specifically the areas to be addressed in the 
DSS are style, organization, information access, and visual 
structure of the page and grade level. 

Assumptions: It is assumed that the officers and managers 
in the DoD would welcome the chance to improve the instruction 
manuals currently used by their E-l's - E-5's. Furthermore, 
style checkers such as "Grammatik IV" and "WriteRighter" 
primarily check text for grammatical rules only and not for a 
range of style features, content, information access or visual 
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structure of the page. Therefore, these checkers would have 
limited success in testing the readability and/or 
comprehensibility of instruction manuals. 

E. LITERATURE REVIEW AND METHODOLOGY 

Current material from the document design field and Plain 
English movement will be reviewed to determine the best way to 
structure a written document whose sole purpose is adult 
instruction . 



F. ORGANIZATION OF STUDY 

Chapter I: Introduction 
Chapter II: Literature Review 

Chapter III: Rule Selection and Justification 

Chapter IV: Proposed DSS Design 

Chapter V: Conclusions and Recommendations 
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II. LITERATURE REVIEW 



This chapter examines current literature in the document 
design and Plain English field to determine which current 
practices can be developed into rules for the DSS. 

A. OUTLINE 

The information in this chapter will be presented in the 
following manner: 

1. Two factors critical to the useability of a document 
will be examined. These factors are readability 
formulas and comprehensibility. 

2. The limitations in their applications will be examined. 

3. Critical areas that these formulas do not address will 
be examined for their impact on the adult reader. 

4. Specific strategies will be discussed that make 
documents more readable and comprehensible. 

B. READABILITY FORMULAS 

There are two different areas that must be investigated 
when one studies the useability of instruction manuals: 
readability and comprehensibility. 

Guillemette (1987) defines readability of a text as the 
extent that the intended readers are able to read it quickly, 
accept it (i.e. persevere in reading it), and understand it 
clearly [Ref. l:p. 41]. Readability can be measured by 
mathematical formulas that are applied to text to provide an 
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index of how difficult the text will be for a given group of 
readers. These formulas measure one or two features of a text 
to produce a numerical value between 0 and 100 or a grade 
level. The features most commonly measured are sentence length 
and word length or word frequency. 

Hundreds of formulas have been developed to predict 
readability for different groups and ages. However, Klare 
found that no "best" formula exists. Of the widely used 
formulas, the Dale-Chall formula is the most accurate and the 
Flesch Reading Scale (FRS) is the most popular [Ref. l:p. 43] 
[Ref. 2] . Another formula that is widely used in business and 
government to test text for adults is the Gunning Fox Index 
[Ref. l:p. 46] [Ref. 3:p. 43]. Since these three formulas are 
so popular, an explanation of each one will be listed below. 
The mathematical calculations for these formulas are shown in 
Appendix A. 



1. Flesch Reading Scale: The formula is based on sentence 
length and number of syllables per one hundred words. 
The FRS renders a number between 0 and 100. The lower 
the number the more difficult the passage is to read. 
The results from this formula can be converted to grade 
levels . 

2. Dale-Chall: The Dale Chall formula counts sentence 

length and frequency of whether a words appears on a 
3000 word list of acceptable words. It yields a grade 
level estimate. 

3. Gunning Fox Index: The Fog Index counts sentence length 
and percentage of polysyballic words (words of 3 or 
more syllables) . It also yields a grade level estimate. 
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Readability formulas may be used in two ways. First, they 
are applied to a text after it is written to ensure the text 
is written at a preselected grade level. If the grade level of 
the text is higher or lower then the desired grade level, the 
text is revised accordingly. Second, the formulas can be 
applied in the same manner to each paragraph as it is written 
to provide immediate feedback to the writer. Computer editors 
can be programmed to offer immediate feedback on the 
readability of individual paragraphs or text. "Grammatik IV" 
and "RightWriter" are examples of these types of program. Text 
is first reviewed by an editor. Suggestions for better format 
and grammar are then generated. In addition, the grade level 
of the text is determined. 

The question that should be asked here is whether or not 
people should write to formulas. According to Klare (1979), 
experts in the area of readability agree that it is 
ineffective to write to formulas, that is, to change only 
those features measured by the formulas without regard to 
whether or not the changes made make the , aterials easier to 
understand. He feels the best way to design a text that reads 
at a specified reading level is to use clear writing 
techniques [Ref. 4:p. 125] [Ref. 5]. Klare has developed a 
seven step procedure for using readability formulas when 
designing a document [Ref. 4:p. 125]. 

1. Apply a formula to see if a piece of writing is likely 
to be readable to intended readers. 
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2. If the readability index suggests it is, and if other 
requirements for good writing have been met, stop here. 
In other words, that a poor index value predicts poor 
writing, a good index value by itself need not mean 
good meaning. 

3. If the readability index suggests the piece of writing 
is not likely to be readable to its intended writers, 
put the formula aside so as not to be tempted to "write 
to formula" 

4. Rewrite the material, trying to discover and change 
those parts likely to cause trouble. Use the formula 
information only as a guide to where to begin. 

5. Apply the formula again to see if the piece of writing 
is now more likely to be readable to intended readers. 

6. If it is, and other requirements for good writing are 
met, stop there. 

7. If it is not, repeat steps, 3, 4, and 5 until an 

appropriate readability index is achieved. 



C. COMPREHENSIBILITY 

Comprehensibility is defined as the reader's ability to 
perform a task after reading text describing how that task 
should be done [Ref. 4:p. 117]. Since testing a subject's 
ability to perform a task after reading a text is prohibitive, 
comprehension tests have been developed. There are several 
ways to measure comprehensibility. Two common methods are 
using a Cloze test, or a Cloze test and a reading test. A 
cloze test requires the subjects to replace deleted words in 
text (usually in an nth word deletion) ; a passage with a 
higher mean number of correct responses is considered more 
readable. [Ref. l:pp. 41-42] 
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Several researchers have assigned comprehension scores to 
various passages based on combined reader's performance on a 
reading and a cloze test. For example, Kincaid et al assigned 
comprehension scores based on the results of the cloze test 
and the Gates-MacGinite reading test. They said individuals 
comprehended a passage when they scored 35% or more on a cloze 
test of that passage. To determine the reading grade level 
required for comprehending that same passage, they categorized 
the readers into reading grade levels based on the Gates- 
MacGinite reading test. Then, all the readers who fell in the 
same groups were tested to see if 50% or more of the readers 
in that group comprehended the passage: that is, scored 35% on 
more on the cloze test. 

Caylor et al., used an almost identical procedure to 
determine comprehensibility of a text. Instead of using the 
Gates-MacGinite reading test, they used the Kincaid-Flesch 
formula. According to their test, a reader was said to 
comprehend a passage when 50% or better of the readers in a 
specific grade level scored 35% or better on a cloze test for 
that passage [Ref. 4: p. 122] [Ref. 6] [Ref. 7] [Ref. 8]. Since 
comprehension test are easy to administer and grade and 
individually testing the users of a text is costly , 
comprehension test scores are commonly used as the sole 
measure of comprehensibility. 

Even though readability formulas are widely used because 
of their ease of application, there is widespread disagreement 
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about their value and effectiveness to predict readability and 
comprehension. Redish, Selzer, Klare, Guillemette are a few of 
many researchers who disagree with the ability of reading 
formulas to predict reading levels. Redish and Selzer have 
pointed out five problems with readability formulas. 
[Ref. 3 : p . 47] 



1. Readability formulas have been applied to technical and 
business writing with no research basis. 

2. Studies show that readability formulas are not reliable 
and valid predictors of how understandable a technical, 
scientific or legal document will be for adults. 

3. Shortening sentences and words does not necessarily 
make the sentences and words easier to understand. 

4. The underlying assumption of readability formulas - 
that any text for any reader for any purpose can be 
measured with the same equation-does not mesh with our 
current understanding of how people process 
information . 

5. Readability formulas do not take into account many 

features that are critical to people's ability to 
understand and use documents: i.e. content, 

organization and layout. 



Klare and Guillemette in discussing the limitations of 
readability formulas state that they do not measure the 
effects of the differing purposes, abilities, intelligence, 
background, expectation , maturity or motivation in reading. 
Nor do they measure the effects of format, typography, content 
or difficulty of the text on comprehension [Ref. l:p. 45] 
[Ref. 4:p. 121] [Ref. 2]. Schumachler and Waller state that 
readability formulas are too global a level of information; 
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they provide little help in determining either how to produce 
a well-designed document or how to improve the design of the 
already existing document [Ref. 9:p. 383]. Guillemette states 
that an early comprehensive investigation into adult reading 
led to the identification of four major categories of factors 
considered useful in predicting the difficulty of reading 
materials: format, organization, style and content [Ref. l:p. 
45] [Ref. 10]. Readability formulas, however, do not provide 
an accurate assessment of these factors because they test at 
most two or three stylistic variables. 

Similar doubts about the applicability and useability of 
comprehension tests also exists. Duffy and Guillemette are a 
a few of many researchers who disagree with the ability of 
comprehension tests to predict -comprehension . Duffy contends 
that comprehension test as they are presently designed in 
principle do not test the "reading comprehension skills 
required to use a text on the job or in training." Duffy also 
contends it is impossible to predict the exact 
comprehensibility of the task; "that is, the task used in the 
development of the test, is grossly different from the 
practical tasks for which texts and documents are used [Ref. 
4:p. 118] ." According to Guillemette, one of the major problem 
in using comprehension tests "is deciding upon the nature and 
domain of the questions themselves (i.e., recall vs. 
inference) [Ref. l:p. 41]." It is a common practice to use 
objective type questions (multiple choice) in many 
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comprehension test. Guillemette states that the use of 
objective questions have been criticized "because of the 
possible biasing effects of subject questioning and question 
formulation." [Ref. l:p. 41] 

D. CONTENT, ORGANIZATION, FORMAT AND STYLE 

In this next section the four factors listed above will be 
defined. The impact they have on the readability and 
comprehensibility of a document will be addressed. 



1. Content: Redish and Selzer define content as the 

appropriateness and accuracy of the text. In other 
words, text needs to be designed for its intended 
purpose. This thesis focuses on procedural documents, 
which show an individual how to accomplish a task. 
[Ref. 3:p. 50] 

Guillemette defines content as the theme, nature of 
subject matter, and unity of content. Content factors 
affect the interest and compellingness of the material 
for the reader. [Ref. l:p. 42] 

2. Organization: Redish and Selzer define organization as 
the ease of access to the right information [Ref. 3:p. 
50]. It is the use of access devices (i.e. indexes) and 
reference devices (glossaries) outside of the text of 
a document, and the use of titles, headings, purpose 
statements, topic sentences and internal previews 
within a text. Organizational factors impact 
accessibility of the reading material for the reader. 
[Ref . 1 :p . 42 ] 

3. Format/Layout: Selzer and Redish define format as page 
layout and typography and the use of graphics [Ref. 
3 :p . 50] . 

Guillemette defines format as the size and binding of 
the material; page layout appearance and quality; and 
kind of type and graphical displays. Format factors 
affect the fluency aspect of the reading process. He 
states that fluency can be influenced by a number of 
factors, such as legibility of the basic characters, 
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variations in character form and size, and the special 
arrangement of characters on the page- [Ref. l:pp. 41- 
42] [Ref. 11] [Ref. 12 :pp. 307-340] 

4. Style: Guillemette defines style as semantic and 

syntactic variables, tone of the writer, and the method 
and style of presentation. Linguistic structure or 
style affects reading behavior. [Ref. l:p. 42] 



Specific strategies will now be discussed to design better 
manuals. These strategies consider the factors that affect the 
difficulty of reading material. These strategies will be 
divided into four specific areas: graphics, organization and 
access of information, visual structure of the page, and 
style . 



E . GRAPHICS STRATEGIES 

Graphics - The author of this thesis defines graphics as 
the layout, design and placement of graphs, charts, figures 
and illustrations in the text. 



1. Whenever possible do not present information in prose. 

Research has shown that people extract information more 
quickly and easily when it is presented in lists and 
tables than when it is presented in prose. [Ref. 13:p. 
112] [Ref. 1 4 : pp . 160-61] [Ref. 1 r 

2. Orient charts so that they are read left to right. 

Charts seem to be better processed if they are read 
left to right. This orientation takes advantage of 
"normal" reading habits and improves reading speed. It 
also suggest that charts should be typeset in the same 
direction as the text whenever possible. [Ref. 16:p. 
81] [Ref. 17] 

3. Use a simple line graph if possible. Simple line graphs 
should be used to present a few points. One study that 
treats graphs as a independent element finds that a 
simple line graph works better than bar graphs or pie 



12 



charts when the number of points on the graph are few. 
[Ref. 1 6 : p . 81] 

4. Use colors wisely and correctly in a graph. When using 
colors to illustrate a graph, use only a few colors, 
also, include a legend and use colors in a manner 
consistent with general population stereotypes. Color 
seems to work as a discriminating agent when it 
includes only a few colors and a careful description of 
the rationale for the color code. [Ref. 16:p. 81] [Ref. 
18 : p . 16] [Ref. 19] 

5. Repeat tables whenever they will be used in the text. 

Readers find it more advantageous to have tables 
repeated strategically throughout a text rather than 
having to perform elaborate searches through lists of 
tables and indexes to find an often used table. [Ref. 
14 :pp. 160-161] [Ref. 16:p. 81] [Ref. 20:pp. 331-343] 
[Ref. 21 : pp . 175-187] 

6 . Place columns within a table closer together rather 

than spacing them across a page. This makes searching 
easier and reduces additive operations. [Ref. 14:pp. 
160-61] [Ref. 1 6 : p . 81] [Ref. 20:pp. 331-343] [Ref. 

21 : pp . 175-187] 

7. Use the best type of graphical display for a specific 

purpose. Studies indicate that tables are superior to 
graphs when looking up and recalling specific values. 
They are also superior to graphs when comprehending 
demographic statistics. Conversely, graphics are 
superior to tables for subjects asked to compare data. 
On the other hand. Powers et al finds the combination 
of text and graphics is superior when subjects required 
to make decisions when accuracy is required. 
[Ref. 1 7 : p . 17] [Ref. 22:pp. 361-376] [Ref. 23:pp. 787- 
789] [Ref. 24:pp. 145-58] [Ref. 25:pp. 32-39] [Ref. 

2 6 : pp . 545-566] 

8 . Limit the number of visual variables, i.e. size, 

value, direction, texture, shape, and color when using 
a visual aid. Too many visual cues impede 

comprehension. [Ref. 17:p. 11] [Ref. 27:pp. 595-603] 

9. Be consistent in your organization of graphics from 
section to section and chapter to chapter so that the 
same graphics cues mean the same thing. [Ref. 17 :p. 12] 
[Ref. 28 : p . 27] 
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10. Use only those lines, grid patterns, necessary to make 
the information clear. [Ref. 17:pp. 11-12] [Ref. 27:pp. 

595-603] [Ref. 29:p. 38] 

11. Present "one idea per visual" [Ref. 14 :p. 14] 



F. ORGANIZATION STRATEGIES 

Organization - organization and information access is 
defined as arranging the content in a logical manner so 
information is quickly and easily ascertained. 



1 . Organize the document according to the reader' s stated 
purpose for the text. Distinguish between text used for 
doing and text used for learning. In text used for 
learning, the reader's goals is to absorb the material 
and remember it for future use. In text used for doing, 
the reader' s goal is to read enough to act immediately 
to make a decision or to follow steps in a procedure. 
A procedural text should be designed differently from 
a definitional or recreational text. [Ref. 10:p. 104] 

2. Make the document user-oriented not content-oriented. 

Put yourself in the user's place and ask questions the 
reader is likely to ask. Then order the questions and 
responses logically. [Ref. 10:p. 104] [Ref. 28:p. 20] 
[Ref. 30 : p . 123] [Ref. 31] [Ref. 32] 

3. Include only the information the user needs. One key to 

good business writing is to provide only the 
information that busy people need to accomplish their 
task. [Ref. 13:pp. 104, 110] [Ref. 28:p. 21] [Ref. 33] 

4. Write headings as questions or verb phrases. Research 
supports that nouns by themselves do not help people 
understand how text is organized. Questions and verb 
phrase both work well for informative headings. 
Questions work well for information sheets and 
brochures. Verb phrases work well for procedural 
manuals. [Ref. 13 :p. 107] 

5 . Include a table of contents and an index whenever 
possible. This may be the difference between a useful 
manual and one that sits on the shelf. Index verbs as 
well as nouns and include in the index words that the 
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readers will bring to the document even if those words 
are not in the document [Ref. 13:p. 109] 

6. Set up signposts. At the beginning of a document, set 
the overall context by telling the reader what is in 
it, why they might choose to read parts of it, and what 
you expect them to get from it. When you get down to 
the procedure and details, set the context repeatedly. 
Don't just tell them why and under what circumstances 
they might choose to do that procedure, rather than 
another procedure. [Ref. 28:p. 22] 



G. VISUAL STRUCTURE OF A PAGE STRATEGY 

Visual structure of a page - Visual structure of a page is 
defined as the arrangement and layout of type, words, 
sentences and paragraphs in a text to enhance the appearance 
of the text on the page. 



1 . Use boldface type to emphasize words or short portions 
of text. Research suggests that when a text has a 
logical structure, typographic distinction such as 
changes in types, weight, or typeface may help readers 
understand the structure. Research specifically 
indicates that readers notice changes in type weight 
(heavy, medium, light) more readily than they notice 
changes in typeface and that readers find very light or 
very heavy type tiring and difficult to read. [Ref. 
1 6 :p . 79] [Ref. 29:p. 37] [Ref. 34] [Ref. 35:pp. 57-66] 
[Ref. 36] [Ref. 37] [Ref. 32:p. 79] [Ref. 33] [Ref. 34] 

2 . Use italics and capitalization when one needs to slow 

down reading rates. Research has shown that italics and 
capitalization will consistently slow down reading 
rates. Their usefulness is in cautions, warnings and 
the like. [Ref. 16:p. 80] [Ref. 38] [Ref. 39] [Ref. 

4 0 : pp . 273-280] [Ref. 41:pp. 542-570] [Ref. 42:pp. 137- 
146] [Ref. 4 3 : pp . 541-550] 

3 . Use white space to enhance the appearance of the text . 

Text can be enriched by selectively using certain 
elements that create additional white spaces around 
letters, words, and lines. One can add informational 
value to a text by using selective typography to 
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achieve specific reader reactions, or they can 
judiciously apply certain criterion for additional 
white space such as word and letter spacing, extra 
leading and paragraph leading. [Ref. 16:p. 81] [Ref. 

29:pp. 36, 38] [Ref. 44:pp 90-95] [Ref. 45:pp. 633-640] 
[Ref. 4 6 :pp . 457-471] [Ref. 47] [Ref. 48:pp. 39-42] 

4 . Choose between a serif and sans-serif typeface 
according to the visual tone of the document you want. 

Use a type size of 10 points or larger, two or more 
points of leading between each line, and a moderate 
line length. [Ref. 29:p. 36] 

5. Avoid using all uppercase letters. Research supports 
the conclusion that text printed in lowercase letters 
is faster and easier to read than text set in uppercase 
letters. [Ref. 29:p. 37] [Ref. 49] [Ref. 50] [Ref. 51] 
[Ref. 52] 



H. STYLE STRATEGIES 

Style - style is defined as the writer's selection and 
choice of words, sentence types and structure to make the text 
more readable and understandable. 



1 . Use personnel pronouns 

2 . Use strong action verbs 

3. Include the user in the text. Flower et al "found that 
readers most often converted passive-nominal sentences 
into active-verbal sentences in order to understand 
passages. The researchers coined the phrase "scenario 
principle" to describe text that is "structured around 
a human agent performing actions in a particularized 
situation." The principle has proven to be one of the 
easiest to teach trainees. The principle unites and 
explains several guidelines commonly suggested by style 
manuals, namely use personal pronouns, active voice, 
and use concrete, specific verbs. [Ref. 53:p. 53] [Ref. 
54 :pp . 21-32] [Ref. 55:pp. 41-58] 

4. Use active and passive voice at the appropriate times. 

Active voice works well if the purpose of the document 
is to stress the consequence of an action. However if 
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the receiver is more important than the action, a 
passive construction is often the best choice. [Ref. 
53 : p . 53] [Ref. 30:p. 117] [Ref. 33:pp 5-15] [Ref. 56] 

5. Use Some contractions. Research has shown that readers 
are less likely to see the not when it is not 
contracted. It also generally accepted that the tone of 
a document is less formal when contractions are used. 
[Ref. 57: pp. 1-11] 

6 . Use ordinary words . 

7. Present information in concrete terms. More recent 
research has shown that it is not just the length of a 
sentence that affects how easily readers can understand 
it. Material presented in concrete terms is easier to 
understand than material presented in abstract terms. 
Mclaughlin determined that whereas a less legible, more 
difficult version of a technical pamphlet did not 
impede performance for highly motivated individuals, 
individuals indicated they would never have used it 
except under duress. Less motivated subjects performed 
significantly worse with the more difficult version. 
Simpler reading materials are generally preferred, even 
when the reader is capable of understanding more 
difficult versions of the same material. [Ref. l:p. 41] 
[Ref 1 0 : p . 112] [Ref. 57:pp. 1-12] [Ref 58:pp. 257-259] 
[Ref. 59:pp. 1-5] 

8. Avoid Nominalizations . Flowers, Hayes, and Swarts found 
that readers had problems understanding writing that 
was full of passive sentences with noun strings and 
nominalizations (nouns made out of verbs) [Ref. 3:p. 
49] [Ref. 55:pp. 41-58] [Ref. 57:pp. 1-12] 

9. Write sentences that do not overtax short term memory. 

Propositional density (the number and organization of 
ideas) is more important than the number of words. Even 
in sentences of reasonable length, grammatical 
structure can create comprehension problems. For 
example, sentences with extra phrases at the beginning 
(left branching) and sentences with the extra phrases 
in the middle (center embedded) are more difficult to 
understand than sentences with extra phrases tacked 
onto the end (right branching) [Ref. 53:p. 54] [Ref. 
13:p. 112] [Ref. 60:pp. 491-499] [Ref. 61] [Ref. 62:pp. 
5-8] [Ref. 63:pp. 17-22] [Ref. 64:pp. 292-303] [Ref. 

65 : pp . 512-521] [Ref. 66:pp. 289-296] 

10. Avoid the use of complex conditional sentences. 

Sentences of this type are difficult for readers to 
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understand. Holland and Rose (1981) found out that, 
after a brief initial practice session, people could 
assign themselves the proper condition more quickly and 
more accurately from an algorithm (a paper flowchart) 
than from single prose sentence [Ref. 53:pp. 54-55] 
[Ref. 15] 

11. Line Length. There is a number of opinions on the 
correct line length. According to Tinker (1965), a line 
length of 50 to 70 characters is easiest for the eye to 
scan [Ref. 53:p. 55] [Ref. 38]. According to Frase, 

McDonald and Keenan The mean best line lengths for 
passages in different readability groups were also 
different: 44, 50, and 56 characters for the easy, 

medium and difficult groups respectively. Their data 
suggested that line length between 40 and 60 are 
suitable for most texts. [Ref. 67:p. 104] 



The design of a document includes incorporating the 
guidelines presented in each of the four areas to create a 
easily used document. Following the principles outlined above 
will allow a document designer to produce a useable product. 

In the next chapter the characteristics of the target 
audience. Department of Defense (DoD) E-l's - E-5's, will be 
determined. The guidelines presented in this chapter will then 
be reviewed to determine which ones will be used to create the 
rules for the decision support system (DSS) . 
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III. RULE SELECTION AND JUSTIFICATION 



In this chapter the guidelines discussed in Chapter II 
will be reviewed to determine which one will be used as the 
DSS's rules. The information in this chapter will be presented 
in the following manner: 



A. OUTLINE 

1. A profile of the target audience who will use the 
manuals developed from this DSS will be established. 

2. Information about the instruction manuals currently in 
use will be provided. 

3. Poor instruction manual characteristics that have 
caused user difficulties will be discussed. 
Justification for the rules chosen for the DSS will 
also be presented in this section. 

4 . Problems caused as a result of poorly constructed 
manuals will be outlined. 



B. TARGET AUDIENCE PROFILE 

In this section a composite profile of the reading skill 
level and educational background of DoD E-l's to E-5's is 
established. To achieve this composite, the Defense Department 
Management Center (DMDC) provided a listing of Armed Forces 
Vocational Aptitude Battery (ASVAB) Verbal (VE) composite 
scores for enlisted pay grades E-l to E-5 for each of the 
different services and for DoD as a whole. The ASVAB VE score 
is important because it can be converted into a reading grade 
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level (RGL) . The RGL obtained from the VE scores will be used 

to get an idea of the reading level of the typical instruction 

manual user. Water et al state that: 

A common metric of reading ability has been reading grade 
level (RGL) . A RGL scale is developed by administering a 
reading test to students at every grade level (1st through 
post-high school) . The RGL scale points are then 
determined in reference to the average score within each 
grade, as reflected by school year and month (e.g. a RGL 
of 9.2 refers to the second month [October] of the ninth 
grade). [Ref. 68 :p. i] 

The RGL was used as the sole metric to measure reading 
ability. In October 1988, Brain K. Waters et al coauthored a 
report entitled " Estimating the Reading Skills of Military 
Applicants: Development of an ASVAB to RGL Conversion Table. 
[Ref. 62] 

The objectives of this study were to: 



1. Measure the reading ability of military applicants 
using six reading tests. 

2. Select one ASVAB composite (anchor) with which to 
equate the reading tests. 

3. Equate each reading tests to the anchor. 

4. Recommend a RGL scale for DOD reporting purposes. [Ref. 
68 :p . i . ] 



In the study conducted by the authors, five ASVAB composites 
and six reading tests were examined. The five ASVAB composites 
and six reading tests used in this study are listed below. 
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Composite Name 



ABBREV. 



Subtests in 
Composite 



Armed Forces Qualification Test-Old 
Armed Forces Qualification Test-New 
Verbal 

General-Technical 

Verbal (DOD Student Testing Program) 



AFQT-0 AR+PC+WK+NO/2 
AFQT-N AR+MK+2 (PC+WK) 
VE WK+PC 

GT AR+WK+PC 

VRB WK+PC+GS 



Reading Test Level 

The Adult Basic Learning Examination 3 
Air Force Reading Abilities Test 
Gate MacGinite Reading Test F 

Nelson-Denny Reading Test -- 

The Test of Adult Basic Education A 

Stanford Test of Academic Skills 1 



Form For Grades 

E 9-12 

A AF Enlistees 

1 10-12 

E 9-12 

5 8.6-12.9 

E 8-12 



AR Arithmetic Reasoning 

30 items that require examinees to solve word problems 
typically involving simple calculations. Time limit is 36 
minutes 

PC Paragraph Comprehension 

Examinees read several short paragraphs and answer 15 
questions that assess their understanding of what they have 
read. Time limit is 13 minutes. 

WK Word Knowledge 

35 items that require examinees to select the correct 
meaning of the word or to identify a synonym. 

Time limit is 13 minute. 

NO Numerical Operations 

Examinees are given 3 minutes in which to solve 50 items 
involving simple calculations. The test is designed to 
measure calculation speed. 

MK Mathematics Knowledge 

25 items that measure examinees knowledge of high school 
level mathematics (algebra, geometry, elementary trigonometry) 
Time limit is 24 minutes. 



GS General Science 

25 items that measure examinees knowledge of the biological 
and physical sciences. Time limit is 11 minutes. 

[Ref. 68 : pp . 16, 17, 19] 

The methodology of the study was as follows: 

Twenty thousand, four hundred and twenty-two applicants 
for military service were administered one of the six 
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published reading tests, along with the Armed Forces Vocational 
Aptitude Battery (ASVAB) . Tests were given at 15 Military 
Entrance Processing Stations and their associated 214 Mobile 
Examining Team Sites during a six-week period in spring 1987. 
Order of the ASVAB or reading test administration was 
counterbalanced across testing sessions. [Ref. 62:p. i] 

Study Conclusions were : 

1. The median RGL of military applicants is 10.9 using the 
proposed DOD RGL scale. 

2. ASVAB VE (WK+PC) is the best anchor test for equating 
ASVAB to the reading tests. 

3. The distribution of Air Force Reading Abilities Test 
(AFRAT) total reading scores was closest to the 
distribution of ASVAB VE of the reading tests in the 
study. [Ref. 68:p. ii] 

The final products of the study are raw conversion tables 
which provide a single DOD RGL scale for each of five reading 
tests . 

Even though DoD commissioned the study, the RGL scale this 
study developed has not been officially approved by DoD. The 
(RGL) scale developed in this study will be used to convert 
ASVAB VE composite scores into RGLs because it makes use of 
readily available data on the E-l's to E-5's currently in the 
DoD. The listing of ASVAB VE scores broken down by rank and 
service complied by DMDC is in Appendix B. This listing also 
breaks down the target population (E-l to E-5) into high 
school and non-high school graduates. 
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Table I, Total DoD Members at Different VE Scores is a 
listing of the total number of enlisted service members E-l to 
E-5 in each VE level, by individual service. The purpose of 
this table is to show the number of enlisted service members 
at each different RGL. The RGL that these scores corresponds 
to are also displayed in this table. At the bottom of this 
table the percentage of high school graduates in each service 
is displayed. The percentage of high school graduates was 
obtained by dividing the total number of high school graduates 
in a category by the total number of service members in that 
category . 

Table II, Average VE Scores for DoD Members lists the data 
used to find the average VE score and the RGL for each 
service. The purpose of this table is to show the numerical 
calculations used to determine the average VE score for each 
of the different services. The averages were obtained by 
multiplying the ASVAB VE score by the total number of service 
members with that score. The sum of these multiplications were 
divided by the total number of service members in that 
category . 

The data in Tables I and II are significant because they 
enable us to determine the RGL for E-l's - E-5's in the DoD 
and use this RGL as the level at which DoD instruction manuals 
should be written. VE scores for DoD as a whole are presented 
graphically in Figure 1. 
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I'M) DoD ASVAB VI; Scores 




ASVAB VI; Scores 

Figure 1 . 1990 DoD ASVAB VE Scores 

In 1980 a vocational aptitude battery, ASVAB, was given to 
a nationally representative sample of nearly 12,000 young men 
and woman between the ages of 16 to 23. The sample contained 
approximately equal proportions of males and females, 
including individuals from urban and rural areas, and from all 
major census regions. This analysis, conducted by the Office 
of the Assistant Secretary of Defense, focused only on young 
civilians who were 18 to 23 years of ao« at the time of 
testing. The results of the testing was put-ished in the 1982 
publication "Profile of American Youth." According to the 
profile, the mean RGL was 9.4, the median RGL was 9.6, and the 
standard deviation was 2.41. [Ref. 69:p. 82] 

The RGL's obtained from this study and the analysis of the 
data in Tables I and II are presented below. The RGL's for DoD 
service members and the civilian population are 
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TABLE I . TOTAL DoD MEMBERS AT DIFFERENT VE SCORES 



VE 


Army 


Navy 


MC 


AF 


DoD Total 


DoD RGL 


20 


15 


8 


5 


0 


28 


< 2.8 


21 


22 


15 


4 


2 


43 


<2 . 8 


22 


1 


0 


0 


0 


1 


<2 . 8 


23 


2 


3 


0 


0 


5 


<2 . 8 


24 


2 


0 


0 


0 


2 


<2 . 8 


25 


0 


1 


1 


0 


2 


2 . 8 


26 


4 


4 


0 


0 


8 


3.1 


27 


18 


5 


0 


1 


24 


3.4 


28 


19 


8 


2 


3 


32 


3.7 


29 


42 


18 


1 


2 


63 


4.0 


30 


79 


30 


6 


0 


115 


4.3 


31 


115 


62 


9 


5 


191 


4 . 6 


32 


193 


79 


11 


8 


291 


4 . 9 


33 


309 


158 


16 


11 


494 


5.2 


34 


543 


273 


31 


19 


866 


5.5 


35 


591 


362 


45 


36 


1034 


5.8 


36 


993 


548 


94 


67 


1702 


6.1 


37 


2461 


1457 


273 


256 


4447 


6.4 


38 


2351 


1564 


304 


231 


4450 


6.7 


39 


3783 


2787 


574 


364 


7508 


7.0 


40 


3938 


3035 


736 


568 


8277 


7.3 


41 


5256 


3981 


1010 


773 


11020 


7 . 6 


42 


6286 


4383 


1314 


1279 


13717 


7.9 


43 


7562 


6011 


1834 


1780 


17187 


8.2 


44 


13676 


10680 


3350 


2962 


30668 


8.5 


45 


11879 


9079 


3318 


3460 


27736 


8.8 


46 


13569 


10364 


4049 


4819 


32801 


9.1 


47 


14727 


10892 


4482 


5963 


36064 


9.4 


48 


23066 


16810 


7485 


9836 


57197 


9.7 


49 


18369 


13331 


6004 


9457 


47161 


10.0 


50 


23357 


16448 


7829 


12003 


59637 


10.3 


51 


20479 


14642 


7138 


12480 


54739 


10.6 


52 


21100 


15170 


7546 


13631 


57447 


10.9 


53 


30805 


21733 


11021 


19782 


83341 


11.2 


54 


29837 


22737 


10743 


24280 


87597 


11.5 


55 


22890 


17408 


8148 


16310 


64756 


11 . 8 


56 


25926 


19823 


9129 


18255 


73133 


12 . 1 


57 


33611 


26250 


11641 


24006 


95508 


12 . 4 


58 


21855 


17659 


7439 


16380 


63333 


12.7 


59 


23313 


18892 


7470 


17554 


67229 


> 12.9 


60 


19771 


16608 


6089 


14927 


57395 


> 12.9 


61 


15706 


13074 


4394 


11768 


44942 


> 12.9 


62 


8855 


7570 


2221 


6603 


25249 


> 12.9 


Total 














427376 


324417 


135766 


249881 


1137440 
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TABLE II AVERAGE VE SCORES FOR DoD MEMBERS 



VE 


VE * ARMY 


VE*NAVY 


VE*MC 


VE*AF 


VE*DoD 


.'0 


300 


1 30 


100 


0 


560 


r.i 


•K>2 


315 


34 


42 


003 


\ > 


X X 


0 


0 


0 


22 




•4 4.' 


30 


0 


0 


115 


:a 


•5 3 


0 


0 


0 


48 




0 


25 


25 


0 


50 


> 

. o 


10-5 


104 


0 


0 


20 $ 


~\ • • 


•5 3 3 


135 


0 


27 


64 S 


X v> 
. £> 


530 


"x O X 
— M 


53 


34 


396 


^ ° 


1 0 1 3 


5 5 5 


2 0 


53 


1827 


30 


0370 


900 


130 


0 


3450 


31 


35 35 


1922 


27 0 


155 


5921 


■x X 

X ^ 


cl76 


'X r x .X 

— 5 — b 


352 


25 3 


9312 


X X 
X X 


1010" 


5214 


523 


333 


16302 


X t 

a -i 


1 3-5 30 


°230 


1054 


34 3 


29444 


3 b 


20 333 


1 2 3" 0 


1575 


1230 


36190 


\ 

a o 


35"-! 3 


103723 


3334 


2412 


61272 


v > 


0105" 


5 3 0 0 0 


10101 


0 4 "2 


164539 


3S 


3 0333 


504 32 


11552 


3773 


169100 


30 


1 -5 " 5 3 " 


103303 


223S3 


14103 


292812 


4 0 


15"520 


121400 


2 0440 




331080 


4 1 


015403 


133221 


4141 0 


31303 


451820 


* 7 


0 340 10 


003103 


55133 


53713 


576114 


1 > 

•< 0 


305133 


0534 "3 


"8 3 c2 


7 3540 


739041 


•< •< 


301" -5 -5 


430020 


1 4 " 4 0 0 


130323 


1340392 


45 


534555 


4 03:> 5 5 


140310 


155 "00 


1243120 


4 3 


c 041" 4 


4 " 0 " 4 4 


133254 


22 1 3"4 


1 5 0 8 S 4 6 


M 


c 0 0 1 0 0 


511024 


010 354 


230231 


1695003 


4$ 


110" 1 O s 


303330 


350030 


M ~ i -• O 


2"45453 


3 0 


00003 1 


353010 


004103 


4 0 3 3 9 a 


2310889 


50 


1 1 0 "350 


300400 


301450 


300150 


2081850 


51 


1044400 


" 4 3 " 4 2 


334033 


333430 


2" 01 380 


f X 

S 


1 09 '000 


3 3 34 0 


300302 


"03312 


2 9 8 " 2 4 4 


33 


» xx • * r 

1 OO. P O 0 


1 ’ 5 1 3 4 0 


53411 3 


104344c 


4417Q73 


5 m 


1 6 1 i i 9 S 


102""03 


530122 


1311120 


• — ^V /X "X 'X .X 

1 otLob 


55 


1 053 050 


0 5' "* 4 4 0 


44^140 


$07050 


3561580 


5 0 


1451353 


1 1 10033 


511224 


1022230 


4005448 


5- 


101330 ' 


1403250 


' » x r -x "• 

JJX jx x 5 X 


1333342 


5 4 4 3 95 6 


5? 


103"300 


1024222 


M O 1‘tlV 


050040 


3 3 ^ 3 3 1 4 


5 5 


1 3 1 5467 


1 1 14323 


4 4 0 " 3 0 


1035333 


3966511 


50 


1 1 3 3 0 3 0 


0 0 c 4 3 0 


335340 


$05320 


2*1*10 0 0 


5 1 


0530 33 


" 0 " 5 1 4 


2 3 3 0 3 4 


"1"343 


2^41462 


50 


540010 


4 3 0 3 4 0 


13"" 02 


400333 


156543S 


Tot a 1 


003333 0 3 


1 '052 055 


"131033 


1 3 5 4 " 1 3 4 


©014$$$$ 



- c 



YE Avg. 52.34 52.56 52.90 54.21 52.88 

RGL 11.0 11.1 11.2 11.6 11.2 

presented side by side to provide the reader with a basis with 
which to compare the reading level of DOD service members. 





ARMY 


NAVY 


MC 


Air F 


DoD 


Youth 


VE Average 
RGL 


52.34 
11 . 0 


52.56 

11.1 


52.90 

11.2 


54.21 
11 . 6 


52.88 

11.2 


9.4 


% HS GRAD 


92 . 67 


92 . 04 


93.46 


99.55 


94.09 


59.96 


The DoD RGL 


of 11 . 


2 corresponds 


closely 


to the 


10.9 : 



average obtained by the Brain K. Waters et al study. 

To conclude, the average RGL for E-l to E-5 in the DoD is 
11.2, and 94.09% of the members in this group are high school 
graduates . 

The question here is why the readability level of E-l's to 
E-5's is being presented if there are doubts about the ability 
of readability formulas to accurately predict the readability 
of a document. The answer is that readability formulas and 
comprehensibility scores have value if they are not the sole 
criteria on which the readability and useability of a text is 
based. As indicated in Chapter two "experts in the area of 
readability agree that it is ineffective to write to formulas, 
that is, to change only those features measured by the 
formulas without regard to whether or not the changes made 
make the materials easier to understand [Ref. 4:p. 125] [Ref. 
5] ." If the reading level of a targeted audience is known (as 
discussed above) and document designers desire to write to 
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that reading level, documents should first be checked for the 
other factors that affect its readability and useability. Then 
the reading level of the document should be adjusted up or 
down to reach the desired reading level. These factors as 
identified in this thesis are graphics, organization and 
information access, visual structure of a page, and style. 

C. RULE SELECTION AND JUSTIFICATION 

In this section, justification for selecting and not 
selecting individual guidelines will be provided. 
Justification is based on: 



1. The profile of the target audience (discussed above). 

2. Characteristics of manuals currently in use. 

3. Poor instruction manual's characteristics that have 
caused user difficulties. 

4. The difficulty experienced by the document users as a 
result of poor instruction manual characteristics. 

5. Current practices in the Plain English and document 
design field. 



1. Characteristics of Instruction Manuals Currently in 
Use 



1. Their purpose is to provide listings of the steps 
necessary to complete a task, which the user should be 
able to learn from quickly. 

2. They are normally a collection of instructions 
detailing how to complete a number of various tasks. 
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3 . 



Length is variable, and size is proportional to the 
number of tasks being described. Manuals are generally 
longer than 25 pages. 

4. They are generally narrative in nature. 

5. They normally include some type of illustrations 
pertinent to the tasks being described. 

6. The listing of steps and procedures should not be so 
voluminous as to discourage use. 

2. Poor Instruction Manual Characteristics That Have Led 
to User Difficulties 

The poor instruction manual characteristics that have 
caused user difficulties are listed below. The guidelines from 
Chapter II which correct /address a specific characteristics 
and serve as justification for the rules used in the DSS, are 
listed immediately below the characteristics it references. 



1. Official titles and terms or complex words that refer 
to ordinary tasks that are commonly referred to in 
colloquial terms. 

a. Use personal pronouns 

b. Use strong action verbs. 

c. Use ordinary words 

d. Present information in concrete terms. 

1 . Research has shown that readers often convert 
passive-nominal sentences into active-verbal 
sentences in order to understand passages. 
Researchers coined the term "scenario 
principle" to describe text that is 
"structured around a human agent performing 
action in a particularized situation." This 
principle unites and explains several 
guidelines commonly suggested by style 
manuals, namely use personal pronouns, write 
in active voice, and use concrete, specific 
verbs. [Ref. 4:p. 125] [Ref. 5] 
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2. If information is presented in the simplest 
and most direct manner, information can easily 
be obtained from text. 

3. Presenting information in this manner will 
lessen the chance that needed information will 
be misunderstood by enlisted service members. 

4. If information is easy for your troops to find 
and use, it will be used. 

2 . Too many new ideas or concepts are presented in a 
sentence . 

a. Write sentences that do not overtax short term 

memory . 

1 . Research has shown that it is easier to 
process information presented in concrete 
terms than in abstract terms. 

2. If sentences are poorly constructed, their 
meanings may be obscure. This may mean that 
troops will not be able to extract the 
information they need to complete a task. 
Simple direct sentences are the best why to 
present information. 

3 . Too much needless information about a concept or idea 
is presented in a sentence or paragraph. 

4 . Terms and concepts presented in the manual are not 
defined before they are used. 

a. Include only the information the user needs 

1. If superfluous informatic „ is presented in a 
manual, there is a change that it will be 
acted upon. 

2. If information is kept simple, there is little 
chance that it will be misunderstood. 

3. Presenting only the necessary information in a 
manual saves troops time, as their time is not 
taken up weeding through useless information. 

5 . Directions for completing a task that require the user 
to evaluate the status of a condition (s) is presented 
in a sentence or paragraph vice a flowchart. 
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a. Avoid the use of complex conditional sentences. 

1 . Research has shown that complex sentences with 
a lot of prepositional phrases are difficult 
to understand. 

6 . Directions for completing a task that require the user 
to perform a series of steps is presented in a sentence 
or paragraph vice a step-by-step listing. 

a. Whenever possible do not present information in 
prose . 

1. Research has proven that it is quicker and 
easier for people to read lists than prose. 

2. By presenting information in a step-by-step 
format. The OIC/NCOIC can ensure that all the 
needed steps in a task are completed in the 
right order. This ensures that E-5's and below 
know exactly what actions to perform and in 
what order to perform them to complete a task. 

7. Directions and the tables and graphs that support those 
directions are not located in the same place in the 
manual . 

a. Repeat tables whenever they will be used in the 
text . 

b. Place columns within a table closer together 
rather than spacing them across a page. 

1 . Research has proven that people would rather 
have tables repeated in a text vice having to 
search for ones previously displayed in the 
text . 

2. Research has proven columns in a table that 
are closer together make searching for 
information easier. 

3. When information is repeated where it is to be 
used, it is very easy for enlisted service 
members to have the correct information when 
they need it. Requiring troops to go back and 
search for previously referenced information 
may make completing a task difficult. It is 
hard to follow instructions one place in a 
manual when its complementary data is 
somewhere else in the manual. 
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8 . Directions are provided, but there are no illustrations 
or they poorly support the directions . 

a. Use the best type of graphical display for a 

specific purpose. 

1. Research has proven that different types of 
graphs are better for processing certain types 
of information. 

a. Tables are superior to graphs for looking 
up and recalling specific values and 
comprehending demographics statistics. 

b. Graphs are superior to tables for comparing 
data Tables and graphs are superior to text 
when making decisions where accuracy is 
required. 

2. Using graphs and or tables for the specific 
tasks it complements, will enable E-5's and 
below to extract the correct information 
qicker and easier, thus making it easier for 
them to complete an assignment. 

9. There is no table of contents 

a. Include a table of contents and an index 

b. Set up signposts 

1. A document is easier to use if the reader 
knows the direction the text will take 
throughout the text. 

2. E-5's and below will be more apt to use a 
manual if the information is easily found and 
accessed. If the information is hard to find, 
it will not be used. 

10. There were no headings for paragraphs that introduced 
new ideas or for paragraphs that linked together 
different aspects of the same idea. 

a. Write headings as question or verb phrases. 

1 . Research shows that headings as questions or 
verb phrases helps the user understand the 
text . 

2. Making the information easy to find or use 
will encourage enlisted service members to use 
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it, since they will not get discouraged 
looking through excessive information. 

11. Crucial Information about completing a task which the 
user must know before attempting to perform that task 
is presented in the same manner and format as 
noncrucial information. Crucial information is not 
emphasized, set aside, or highlighted. 

a. Use boldface type to emphasize words or short 
portions of the text. 

1. Research shows that highlighting text makes it 
stand out. 

2. Boldface type can be used to emphasize the 
information that the OIC/NCOIC feels is very 
important and needs to stand out. 

b. Use white space to enhance the appearance of 
text . 

1. Research shows that value can be added to 
information by selectively using white space. 

2. White space can be added to words, sentences, 
and paragraphs to make them easier to read or 
stand out. Therefore, white space can be added 
to words, etc. to emphasize them or make them 
stand out. 



Poorly designed manuals cause problems that will impact on the 
users ability to complete the mission. The problems caused as 
a result of the difficulties users experienced from poorly 
constructed instruction manuals are listed below. 

3. Problems Caused as a Result of Difficulties Users 
Experienced From Using Poorly Constructed Manuals 

1. Tasks were completed incorrectly. 

2. Tasks simply were not done. 

3. Inordinate amounts of time were spent completing tasks. 
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Tasks were completed only after going to other 
instruction manuals or having a knowledgeable person 
demonstrate how to complete the task. 



4 . 



The guidelines established in Chapter two that will serve as 
rules for the DSS have been justified above. The guidelines in 
Chapter two that were not presented in the previous section 
were not selected as rules for the DSS; they did not 
specifically address/correct poor instruction manual 
characteristics that have caused users difficulties. These 
guidelines are listed below. 



D. NON SELECTED GUIDELINES 

1. Graphics (Nonselected Guidelines) 



1 . Use colors wisely and correctly in a graph 

2. Be consistent in your organization of graphics from 
section to section and chapter to chapter so that the 
same graphics cues means the same thing. 

3. Use a simple line graph if possible 

4. Limit the number of visual variables, i.e. size, value, 
direction, texture, shape, and color when using a 
visual aid. 

5. Use only those lines, grid patterns, necessary to make 
the information clear. 

6. Present one idea per visual 



2. Organization (NonSelected Guidelines) 



1. Organize the document according to the reader's stated 
purpose for the text. 

2. Make the document user-oriented not content-oriented. 
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3. Visual Structure of a Page (Nonselected Guidelines) 

1. Choose between a serif and sans-serif type face 
according to the visual tone of the document of the 
document you want . 

2. Use italics and capitalization when one needs to slow 
down reading rates . 

3. Avoid using all uppercase letters 

4. Style (Nonselected Guidelines) 

1 . Include the user in the text 

2. Avoid Nominalizations 

3. Use active and passive voice at the appropriate times. 

4 . Use some contractions 

5 . Line length 

The author of this thesis does not feel she should 
prioritize the difficulties encountered by manual users 
because she cannot adequately predict what problems different 
users feel are the most crucial. 

In this chapter the guidelines used and not used in the 
DSS were justified. In the next chapter, the system's design 
will be discussed. 
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IV. PROPOSED DSS DESIGN 



In this chapter a proposed design for the DSS is 
presented . 

A. OUTLINE 

The information in this chapter is presented in the 
following manner: 



1. There will be a brief discussion of the cognitive 
factors that effect user interface. Justification of 
the system design based on these factors, will also be 
provided. 

2. The hardware constraints of this system will be listed. 

3. The DSS's design will be illustrated using computer 
screens . 



B. COGNITIVE FACTORS THAT AFFECT THE DESIGN OF USER INTERFACE 

There are several learning phases an individual goes 
through when he/she learns a new skill. Fitts and Posner 
(1967) have termed these phases, cogniti' 1 t phase, associative 
phase and autonomous phase [Ref. 70: p. 169] [Ref 71]. They 
are explained below. 



1. Cognitive Phase: In this phase the beginner attempts to 
understand the skill and is able to crudely perform the 
task using existing habits. Verbal mediation, 
rehearsal, and extensive feedback are characteristics 
of this phase. 
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2. Associative Phase: In this phase the user attempts to 
learn the components of the new skills. He or she tries 
various actions and gradually eliminates the ones that 
are inappropriate. Because the skills are being 
refined, the need for verbal mediation and low-level 
feedback are reduced. 

3. Autonomous Phase: The user further refines the skills 
he/she has learned, until the processes become 
increasingly automatic, and less subjective to 
interferences from other tasks. In this phase the user 
may not need verbalization or feedback. 



In the cognitive and associative states of skill acquisition 
a person learns new skills, and refines or reorganizes 
existing skills. As part of this progression, "the nature of 
a person's knowledge appears to change from an explicit 
verbalizable form to an implicit automatically accessed form 
[Ref 70:p. 170]." These forms of knowledge have been termed 
declarative and procedural [Ref. 70:p. 170] [Ref 72] [Ref 73] 
[Ref 74]. Declarative knowledge is characterized by the fact 
that one is required to interpret and process new facts that 
have been articulated to them. Procedural knowledge is 
characterized by the fact that one accesses direct knowledge 
of physical phenomenon or mental processes. The processing of 
declarative knowledge requires much use of working memory 
because the interpretation and manipulation of knowledge is 
under control rather than occurring automatically. [Ref. 70:p. 
174] [Ref. 75] . 
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C. SYSTEM DESIGN JUSTIFICATION 



Because one is unfamiliar with the material being 
presented and feedback is constantly needed, this process is 
slow and laborious. Because the user will only use this system 
when he/she is developing an instruction manual, it is 
anticipated that their skills will never become automatic. 
Therefore the system is designed as if the user is in the 
cognitive learning phase and manipulating declarative 
knowledge. Comprehensive instructions are given, extensive 
feedback is provided, and minimal computer skills are required 
to use the system. This is evidenced by: 

1. The extensive guidance provided in Figures 2-4. 

2. The simple layout and design of the rule selection 
screens Figures 5 - 8 . 

3. The layout and guidance provided in the recommendation 
screens. Figures 10-14. 

D. SYSTEM HARDWARE CONSTRAINTS 

The system is designed to be displayed on a 19" full page 
monitor. This is a very important requirement because visual 
structure of a page is one of the variables that the DSS 
checks pages of proposed manuals against. An individual using 
the DSS will not be able to adequately evaluate the way 
information is presented on a page if information is shown on 
a space 1/2 to 1/3 the size of a page. A page's dimension are 
defined as 8 1/2" by 11". 
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E. PROPOSED SYSTEM DESIGN 



The system is designed to provide feedback after the user 
has completed a proposed page in the manual. After a page has 
been completed, Figure 2 will appear. It tells the user about 
general instructions. After Figure 2 is displayed, Figure 3 
will be displayed, telling the user how the feedback will be 
displayed. It explains what types of advice the system 
provides and allows the user to select the type of advice they 
prefer [Ref. 76:p.49]. To select the type advice to be given, 
the user chooses a critic. The critic used in Figures 10-14 is 
active/positive. After the user chooses an expert, Figure 4. 
will appear on the screen. It tells the user what factors the 
DSS uses to evaluate the proposed page. It allows the user to 
evaluate their proposed page using: 

1. all the rules for all five variables 

2. selected rules for a single variable 

3. selected rules for a combination of variables 

This DSS can check each page for all five variables in 
tandem or it can check the page one variable at a time. If the 
second method of evaluation is chosen, the systems's 
recommendations will be displayed one factor at a time, in the 
order the factor's where chosen by the user. The evaluations 
are iterative, in that each subsequent evaluation incorporates 
the previous recommendations. The order the factors were 
chosen in this thesis are graphics, style, organization, 
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visual structure of the page. After the user selects the 
factor he/she would like the page evaluated for. Figures 5-8, 
the rules for each factor, will appear. These screens allow 
the user to select the rules they want the DSS to use when it 
evaluates a page. 

Figure 9 is the user's proposed page design. Figures 10 - 
14 display the system's recommendations as it goes through the 
iterative evaluation process. 

The user has the option to save the changes, print the 
changes, to precede to the next evaluation, return to the rule 
selection screen or exit the program, after each evaluation. 

The important feature of this DSS is its ability to 
provide recommendations the user should follow. The system 
provides recommendations only; -it does not arbitrarily change 
the user's text or design. 

In this chapter the design for the proposed DSS was 
displayed. The final chapter will be conclusions and 
recommendations for the entire thesis. 
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General DSS Instructions 



General instructions for exiting the program and moving between the 
levels of the program are presented below. These instructions can 
be used at any level of the program. 



To return to a previously selected screen/level 

1. Push the ESC Key 

2. If you are at this screen and hit the ESC key 
you will EXIT out of the program. 

To Exit out of the program from any place in the program 
1. Hit SHIFT F2 



To use previously selected rules from your last session 

1. Hit SHIFT F3 

2. The rules you choose to use for a session are automatically 
saved when you exit the program. 

3. When you push SHIFT F3 here, you go automatically to the 
recommendation screens . 



To continue with the program hit any KEY. 



Figure 2 . General DSS Instructions 
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This DSS is designed to provide recommendations about a 
proposed page of the manual, after you complete that page. 

The feedback will be presented in the following manner, 
two pages will be displayed side by side. 



The Left Hand Page - Will be the user's proposed 

page design. 

The heading on the TOP of this 
page will read User's Proposed 
Page Design. 



The Right Hand Page- Will be the critic's recommended 

page design. 

The heading on the TOP of this Page 
will read Critic's Proposed Page 
Design. 



This DSS has four types of critics, that will be explained 
below . 

The user can select the type of advice he or she would like to 
receive, by activating that critic. 

To activate a critic: 

1. Move the cursor to the desired critic 



2. Press the ENTER key. 



Critics 



Reactive: a reactive critic will make comments about what the 
user has done. 



Active: an active critic suggests what the user might do or 

proposes criteria the user should consider 

Positive: a positive critic praises a supe 9 r design or 

complains about an inferior design. 

Negative: a negative critic complains about unsatisfactory 
designs and does not praise useful or interesting 
designs . 



Reactive 


1 Active 




Positive | 




Negative 


i i 


11 




i i 


U I| 



Figure 3. DSS Introductory Screen 
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This DSS provides advice on five factors that affect the 
design of a manual. 

The DSS can check each page for all five factors, but it can 
only check the page one factor at a time. 

However when you select a factor you can select more than one. 
The DSS will check for each factor separately, in the order you 
selected the factors. There will be a separate page of 
recommendations produced for each factor. Each subsequent 
recommendation will incorporate the previous recommendations. 

There will be a final recommendation that incorporates 
all the recommended changes. 

Graphics: The layout, design and placement of graphs, 

chart s, figures and illustrations in the text. 



Organization: Arranging of the content in a logical manner 

so information is quickly and easily ascertained. 

Visual Structure of the Page: The arrangement and layout of 

type, words, sentences and 
paragraphs in a text to enhance the 
appearance of the text on the page. 

Style: The writer's selection and choice of words, 

sentences, sentence type and structures to make the text 
more readable and understandable. 

Grade Level: is the grade level at which the document is written 
this will automatically appear as the last 
recommendation . 

Final All: produces a version of the page that incorporates 

the recommendations from all the rules in all 
factors . 

Final Select: produces a version of the page that incorporates 

the recommendations for only the factors and rules 
the user selects. 



Graphics 




Organization 


Visual Structure of Page 










■ 

Style 


1 


” " “II 

Final All 


n H 

||Final Select 



Figure 4 . DSS Factor Selection Screen Graphics Rules 
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Graphics Rules 



The rules the DSS uses to evaluate the graphical quality of a 
proposed page are listed below. You can choose the rules 
you would like the DSS to use. 



How to choose the rules: 

1. Move the cursor to the YES or NO box 
and push the ENTER key. 



a. Choose all the graphics rules 


Yes 




No 


b. Whenever possible do not present 
information in prose. 






Yes 




No 


c. Orient charts so they are read left 
to right . 






Yes 




No 


d. Repeat tables whenever they will be 
used in the text . 






Yes 




No 


e. Place columns within a table closer 
together rather than spacing them 






Yes 1 




No 


across the page. 






f. Use the best type of graphical 
display for a specific purpose 


Yes 




No 










Yes 




No 



I 



Figure 5. 



Graphics Rule Selection Screen Organization Rules 
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Organization Rules 



The rules the DSS uses to evaluate the organization of a 
proposed page are listed below. You can choose 
the rules you would like the DSS to use. 



How to choose the rules: 

1. Move the cursor to the YES or NO box 
and push the ENTER key. 



Choose all the organizational rules 


Yes 


i 


No 

I 


Present only the information the 
user needs. 






Yes 




No 

1 


Write headings as questions 
or verb phrases 






Yes 




No 


Include a table of contents 






| | 

Yes 




No 



e . 


Set up signposts 






i 






Yes 

[ .... . 




No 

_ 



Figure 6. Organization Rule Selection Screen 
Visual Structure of a Page Rules 
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Visual Structure of a Page Rules 

The rules the DSS uses to evaluate the visual structure of a 
proposed page are listed below. You can choose the rules you would 
like the DSS to use. 



How to choose the rules: 

1. Move the cursor to the YES or NO box 
and push the ENTER key. 



a. Choose all the visual structure rules 



b. Use boldface type to emphasize 
words or short portions of 
the text 

c . Use white space to enhance the 

appearance of the text. 



Yes 




No 






Yes 




No 






Yes 




No 



Figure 7 . Visual Structure of a Page Rule Selection 

Screen Style Rules 
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Style Rules 



The rules the DSS uses to evaluate the style a proposed page was 
written from are listed below. You can choose the rules you would 
like the DSS to use. 

How to choose the rules : 

1. Move the cursor to the YES or NO box 
and push the ENTER key. 



a. Choose all the style rules 



b. Use personal pronouns 



c. Use strong action verbs 



d. Use ordinary words 



e. Present information in concrete 
terms . 



f. Write sentences that do not overtax 
short term memory 

g. Avoid the use of complex condtional 
sentences . 



Yes 




Yes 




Yes 




Yes 




Yes 




Yes 




Yes 



No 




No 




No 

] 




No 




— 

No 




No 




No 

i 



Figure 8. Style Rule Selection Screen User's Proposed Page Design 



47 



User' s Proposed Page Design 



To use a checkbook, you need to do several things: The 
money deposited into the account should be recorded in the 
ledger, The payee should be recorded in the ledger, The 
number of the check should be recorded in the ledger, the 
date of the check should be recorded in the ledger. The 
amount of the check should be subtracted from the previous 
balance, the new balance should be recorded in the ledger. 



Figure 9. Proposed Page Design That The DSS Will Evaluate 
Critic's Proposed Page Design (Graphics) 
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Critic's Proposed Page Design (Graphics) 

To use a checkbook, you need to do several things: 

1. The money deposited into the account should be recorded in 
the ledger. 

2. The payee should be recorded in the ledger. 

3. The number of the check should be recorded in the ledger. 

4. The date of the check should be recorded in the ledger. 

5. The amount of the check should be subtracted from 
previous balance. 

6. The new balance should be recorded in the ledger. 
Recommendations : 

1. You should consider presenting the information in a list 
vice prose. 

2. There are no illustrations, you should consider adding 
them . 

3 . The reading level is ninth grade 

4 . To return to the rule selection screen hit shift FI 



Save changes ? 


Yes 




No 


Print changes ? 






Yes 




No 


Check next factor ? 






Yes 




No 


Exit Program ? 






Yes 




No 



Figure 10. Graphics Recommendations 
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Critic's Proposed Page Design (Style) 

To use a checkbook, you need to do several things: 

1 Record the amount of money deposited into the checkbook 

in the ledger. 

2. Record the payee in the ledger. 

3. Record the check number in the ledger. 

4. Record the date the check was written in the ledger. 

5. Subtract the amount of the check, from the previous 
balance . 

6. Record the new balance in the ledger, 
previous balance. 



Recommendations : 

1. You should consider making the sentences active vice 
passive . 

2 . You should consider replacing the highlighted words 
with simpler words. 

3. Words that are both highlighted and underlined are 
misspelled . 

4 . The reading level is ninth grade 



return to the rule selection screen hit shift 


FI 




Save changes ? 


Yes 




NO 


Print changes ? 






Yes 




No 


Check next factor ? 






Yes 




No 


Exit Program ? 






Yes 




No 



Figure 11. Style Recommendations 
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Critic' s Proposed Page Design (Organization) 



How do I use my checkbook ? 



1 Record the amount of money deposited into the checkbook 

in the ledger. 

2. Record the payee in the ledger. 

3. Record the check number in the ledger. 

4. Record the date the check was written in the ledger. 

5. Subtract the amount of the check, from the previous 
balance . 

6. Record the new balance in the ledger, 
previous balance. 



Recommendations : 



1. You should consider writing this sentence as heading. 
You should consider making the heading a questions 

or verb phrase . 

2 . The reading level is ninth grade 

3. To return to the rule selection screen hit shift FI 



Save changes ? 



Print changes ? 



Check next factor ? 



Exit Program ? 



Yes 



No 



Yes No 

'1 jJ 



r 



Yes 



No 






Yes No 



Figure 12. Organization Recommendations 
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Critic's Proposed Page Design (Visual Structure of the Page) 



How do I use my checkbook ? 



1 Record the amount of money deposited into the checkbook 

in the ledger. 

2. Record the payee in the ledger. 

3. Record the check number in the ledger. 

4. Record the date the check was written in the ledger. 

5. Subtract the amount of the check, from the previous 
balance . 

6. Record the new balance in the ledger, 
previous balance. 



Recommendations : 



1. The page layout looks find, there are no recommended 
changes . 

2. The reading level is ninth grade. 

3. To return to the rule selection screen hit shift FI 



Save changes ? 



Print changes ? 



Check next factor ? 



Exit Program ? 



Yes 




No 






Yes 




No 






Yes 




No 






Yes 




No 



Figure 13. Visual Structure of a Page Recommendations 
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Critic' s Proposed Page Design (Final All Recommendations) 



How do I use my checkbook ? 



1 Record the amount of money deposited into the checkbook 

in the ledger. 

2. Record the payee in the ledger. 

3. Record the check number in the ledger. 

4. Record the date the check was written in the ledger. 

5. Subtract the amount of the check, from the previous 
balance . 

6. Record the new balance in the ledger, 
previous balance. 



Recommendations : 



1. This is the final recommendation for the page, 

it incorporates the recommendations for all the rules 
in all five factors. 

2. The reading level is ninth grade. 



Save changes ? 


Yes 




No 


Print changes ? 






Yes 




i 1 

No 


Check next page ? 






Yes 




l 

No 


Exit Program ? 






Yes 




No 



Figure 14 . Final Recommendation That Incorporates 

All The Changes 
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V. CONCLUSIONS AND RECOMMENDATIONS 



A. CONCLUSIONS 

The DSS proposed in this thesis had been designed to 
evaluate instruction manuals for five factors: 

1 . Graphics 

2. Organization 

3. Visual Structure of a Page 

4 . Style 

5. Reading Grade level 

Current literature in the document design and Plain English 
field has been examined, to develop the rules for this system. 
The system allows the user to select: 

1. The type of advice he/she would like to receive 

2. The factors and rules they would like the DSS to use 
when it evaluates an instruction manual 

9 

Finally, the system produces a series of i ^commendations based 
on the evaluation factors and rules the user selected. 

B. RECOMMENDATIONS 

The purpose of this thesis was to propose a design for a 
DSS, which will help document designers develop instruction 
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manuals that are readable and useable. Since the actual system 
was not built, it is recommended that: 

1. A follow on thesis be done to build the system. 

2. Another thesis be done, after the system is built to 
test and revise the system. 

3. After the system is built and tested, that the system 
be distributed throughout DoD. 
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APPENDIX A 



READABILITY FORMULAS 

Flesch-Scale RE = 206.835 - (.865 * wl)-( 

Dale-Chall C50 = 3.6365 + (.1579 * ndw) 

Gunning Fog GR = .4 * (np + si) 

RE = reading ease 

wl = word length (number of syllables per 
si = average sentence length in words 
ndw = percent of words not on dale list of 
gr = grade [Ref. l:p. 43] [Ref 77] [Ref. 
[Ref. 79] 



.015 * si) 
(.0496 * si) 



100 words) 

3000 words 
7 8 : pp . 11-20] 
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APPENDIX B 



DMDC ASVAB VE SCORE 
FOR DoD E-l'S -E-5'S 
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3761 


3371 


5911 

| 
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i 


i 

4 
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4 


1121 
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2981 
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+ 


9931 
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| + 4 4 4 4 4 | 
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1 

139 
1 __ _ 


i 


951 


3781 


7431 


15131 


12541 


1 

37831 
__ 1 


140 

1 


1 


1201 


2081 


7161 


15481 


13461 


39381 
| 


141 

1 


1 


2241 


3211 

1 


10711 

4 _ . 


39901 

. _ _ _ _ _ _ _ + 


16501 


52561 

| 
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I 
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2911 


4301 


13181 


24551 


18121 


62861 

1 


143 

j _ . 


1 


4171 


5321 
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\ 


29431 


20711 

- — . 


75621 
1 


144 

1 


1 


7821 


10751 


30761 


„ — — 

59861 


2757! 


136761 
__ _ | 


145 

1 


1 


9071 


10731 


26521 


46621 


2585! 


118791 
| 


146 

i 


I 


9771 


11561 


30331 


55601 


28431 


135691 
1 


147 

I 


I 


12391 


14001 


336] i 


57941 


29331 


147271 
| 


148 

| 


i 


17301 


20601 


53361 


100841 


38561 


230661 

1 


149 

1 


1 


17591 


19931 


44231 


69521 


32421 


183691 
| 
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1 


25261 


2.9841 


57521 


82911 


38041 


233571 



(CONTINUED) 
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BY SERVICE AND RANK 



ARMY 



PAYGRADES El - E5 



PC 
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i 


1 
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1 

1 


1 

1 


1 

j 


1 

1 


I 

1 


1 

1 


151 

j 


i 

1 


20681 


23371 


48101 


76391 


36251 


204791 
| 


1 52 

1 


1 


20641 


24691 


49661 


79081 


36931 


211001 
1 


153 

| 


1 


24621 


30471 


70241 


130561 
- + - - 


52161 

_ ^ 


308051 


154 

| 


i 


26921 


32401 


66271 


109121 


63661 


| 

298371 
| 


i 55 

1 


1 


23211 


28921 


5652! 

_ f 


81091 

_ + 


39161 

^ 


228901 
1 


i 56 

i 


1 

. -f ^ • 


27961 


32251 


61551 


93171 


44331 

.•--•---j.. 


259261 


157 

| 


1 


29451 


36771 


76861 


131831 


61201 


| 

336111 

j 
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i 


1 
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26381 


5066! 


76861 


43641 


218551 
| 
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1 


21371 


26151 


5466! 


82331 


48631 


233131 


| 
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1 
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21161 


4605! 


67941 


44611 


| 
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| 
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j 


1 
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3270! 


5963! 


4247! 


157061 
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| 


1 
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1282! 


39741 


30381 


1 

88551 
1 
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| 


1 

i 


1 

| 


1 

1 


1 

1 


1 


1 


1 

1 
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j 


i 

1 


13551 
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72841 


j 
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| 
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| 
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j 
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| 


1 
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397841 
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151584! 


808841 
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1 
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434361 


966211 


1629141 


886111 


1 
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| 
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| 
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1 


11 
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41 


41 


91 
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1 


21 


21 




21 


91 
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| 
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1 
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| 


1 


2! 
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132 
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1 


41 
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in 
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1 
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| 
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1 


1 


71 
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451 
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| 


1 


131 
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| 
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135 

1 


1 
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| 
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1 
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| 


1 
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1 
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| 


1 
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7421 


8471 


5781 


27871 
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1 


1 
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4601 


6981 


9401 


6951 


| 
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1 
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I 


1 

4. 


4171 
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10731 
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39811 
| 
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1 


5191 


8431 


11691 


13331 


9741 


48381 

| 
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| 


1 


7571 


10621 


14191 
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11521 


60111 


144 

| 


1 


12461 


19101 


29081 


30841 


15321 


1 

106801 
| 


145 

1 


1 


13381 


17881 


20981 


23061 


15491 


90791 

1 


146 

1 


1 


14911 


19151 


24701 


27001 


17881 


103641 
| 


147 
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1 


15391 


20481 


26311 


27441 


19301 


108921 

1 


148 

I 


1 


21081 
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30561 


43551 


47401 


25511 
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168101 

1 


149 

| 


1 


21241 


25301 


30661 


33401 


22711 


133311 

1 


150 

I __ __ 


1 


27601 


33001 


36891 


40631 


26361 


164481 

| 
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| 


1 


20881 


26191 


32831 


39041 


27481 


146421 

1 
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1 


21611 
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32621 


40541 


29201 


151701 
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1 


23181 
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1 
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| 


1 
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1 
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1 


1 
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1 
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1 


1 
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60721 
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j 
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j 


i 
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91631 
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1 
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4 
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4 


60491 

_ _ 4 _ _ „ 
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51451 


166081 
j 
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| 
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1 


1 TOTAL 


1 
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1 
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1 


11 


31 


81 


91 
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1 
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1 


21 


51 


71 
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l 
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| 


1 
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I 
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| 
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| 


1 
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1 
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i 
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y 
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y 
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1 
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1 
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[ 
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1 


1 
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851 


3321 

4 _ 
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1 
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| 


1 
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| 
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1 


1 
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14661 
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1 
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| 


1 
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1 
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1 


1 
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4 
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1 
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| 
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1 
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1 
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| 
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1 


1 
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8761 
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1 
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| 


1 
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15951 
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75461 

j 
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| 


1 
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| 
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| 


1 
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17251 


107431 
| 
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1 
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(CONTINUED) 
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BY SERVICE AND RANK 
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1 
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1 
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1 


1 

I 


| 
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21081 
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j 
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1 


1 

\ 
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116411 

j 
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I 


1 


5201 
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27921 
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1 
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| 


1 
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| 
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i 


1 
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4. 
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4 
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I 
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1 
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1 
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1 


I 
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1 
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| 


I 

1 


1 

1 


1 

1 


! 


1 


1 


1 

1 

1 
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1 


1 

1 
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1 
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1 
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1 
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1 


IBS GRAD 
| 


1 


97571 


185641 


516521 


300761 


168381 


1268871 

1 


1 TOTAL 


1 


104881 
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544551 


318681 


189171 


1357661 
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AF 

I 



PG 



•I 



I 



I TOTAL I 



IVE 

i 


i 

i 


1 

i 


1 

| 


1 

1 


1 

1 


1 

I 


1 

1 


121 

| 


i 

1 


i 

J 


1 

11 

-j 


1 

11 


1 

.1 

4- _ 


1 

.1 

+ _ . 


1 

21 

1 


127 

j 


i 


.1 


.1 




.1 


.1 


11 


| 

11 

1 


128 

| 


1 


.1 


.1 


. 1 


21 


11 


| 

31 

1 


129 

1 




1 




.1 


. 1 


.1 


11 


11 


1 

21 

1 


131 

| 


1 


.1 


11 


.1 


41 


.1 


1 

51 

1 


132 

| 


1 


.1 


. 1 
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61 
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.1 


31 
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.1 
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4 
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.1 


21 


51 
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1 
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| 


1 


21 


41 
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1 
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1 


81 


51 
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1481 
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1 
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1 


1 


41 
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| 
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1 


1 
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| 
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1 


i 
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4741 
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| 
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j 


1 
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9711 


5441 


17801 
1 
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1 


1 
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29621 

j 
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j 


i 
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8971 


34601 
1 
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1 


1 
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3341 


7521 
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48191 
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| 


1 


2361 
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29821 


13201 


59631 
| 
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| 


1 
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7861 
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16591 
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| 
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1 
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9061 
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45081 
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94571 
] 


1 
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1 


1 
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13841 
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54691 
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1 
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j 


i 
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4. 


124801 
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| 


i 
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29701 
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24321 


136311 
| 
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| 


1 
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20621 


48451 


92381 


26871 


197821 
1 
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1 
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24381 


48101 


102431 


55231 


242801 
| 
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1 


10501 
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30071 


163101 
| 


1 
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i 


1 
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21301 
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1 
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i 
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(CONTINUED) 
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1 
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1 
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1 
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1 
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1 


1 TOTAL 


1 


123331 
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21 


81 
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1 
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I 


1 


81 


81 


23! 
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j 
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1 


1 
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| 
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| 


1 
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1 
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| 


1 
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